Discriminant Kernel Learning Discriminant Kernel Learning via Convex Programming

Regularized Kernel Discriminant Analysis (RKDA) performs linear discriminant analysis in the feature space via the kernel trick. Its performance depends on the selection of kernels. We show that this kernel learning problem can be formulated as a semidefinite program (SDP). Based on the equivalence relationship between RKDA and least square problems in the binary-class case, we propose an efficient Quadratically Constrained Quadratic Programming (QCQP) formulation for the kernel learning problem. We extend these formulations to the multi-class case based on a key result established in this paper. That is, the multi-class RKDA kernel learning problem can be decomposed into a set of binaryclass kernel learning problems which are constrained to share a common kernel. From this decomposition property, a compact SDP formulation is proposed for the multi-class case. Furthermore, it leads naturally to an efficient QCQP formulation for the multi-class case. As the performance of RKDA depends on the value of the regularization parameter, we show that its value can also be optimized in a joint framework with the kernel. Extensive experiments have been conducted and analyzed, and connections to other algorithms are discussed.